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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 


SUBJECT:  Summary  Report  on  the  Audit  of  the  Effectiveness 

of  the  Defense  Acquisition  Board  Process — FY  1988 
(Project  No.  8AE-0051) 


Introduction 


This  final  summary  report  on  the  Audit  of  the  Effectiveness 
of  the  Defense  Acquisition  Board  (DAB)  Process--FY  1988  is 
provided  for  your  information  and  use.  This  report  is  the  fifth 
in  a  series  of  summary  reports  that  have  been  prepared  since  1984 
on  the  executive  level  decisionmaking  processes  that  DoD  used  to 
monitor  and  evaluate  major  acquisition  programs.  This  report 
summarizes  the  results  of  five  audit  reports  that  we  issued  on 
the  DAB  process  from  September  1988  through  July  1989.  The 
objectives  of  the  audits  were  to  evaluate  the  effectiveness  of 
the  DAB  process,  compliance  with  applicable  DoD  Directives  and 
Instructions,  and  applicable  internal  controls. 

Scope  of  Audit 

To  satisfy  the  audit  objectives,  we  focused  on  five  major 
acquisition  programs  that  were  scheduled  for  Milestone  IIIA  (low- 
rate  initial  production)  and  Milestone  IIIB  (full-rate 
production)  DAB  decisions  (Enclosure  1).  The  programs  audited 
were  the  Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM), 
SSN-21  submarine.  Follow-on  Ultra-High  Frequency  (UHF)  satellite, 
C-17  cargo  aircraft,  and  T-45A  trainer  aircraft.  For  each  of  the 
programs,  we  determined  the  completeness  and  accuracy  of 
documentation  that  the  Military  Departments  submitted  to  the  DAB 
to  support  the  scheduled  production  decisions.  We  determined 
this  by  examining  Decision  Coordinating  Papers,  Integrated 
Program  Summaries,  System  Threat  Assessment  Reports, 
developmental  and  operational  test  plans  and  results,  and  other 
supporting  information  specified  in  the  5000  series  of  DoD 
Directives  and  Instructions.  We  used  the  policies  and  procedures 
in  the  5000  series  of  Directives  and  Instructions  as  criteria  in 
determining  the  completeness  and  accuracy  of  program 
documentation. 

As  part  of  the  audits,  we  also  determined  whether  the 
production  rates  that  the  Military  Departments  proposed  to  the 
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DAB  for  the  programs  were  within  the  limitations  specified  by 
Public  Law  and  DoD  Directives  and  Instructions  on  programs  that 
had  not  yet  completed  operational  test  and  evaluation. 

The  five  program  results  audits  included  reviews  of  program 
documentation  dating  from  1980  to  1989.  The  audits  were  made  in 
accordance  with  auditing  standards  issued  by  the  Comptroller 
General  of  the  United  States,  as  implemented  by  the  Inspector 
General,  DoD,  and  accordingly  included  such  tests  of  the  internal 
controls  as  were  considered  necessary.  A  list  of  activities 
visited  or  contacted  during  the  audits  is  in  Enclosure  5. 

Background 

DoD  Directive  5000.1,  "Major  Systems  Acquisition," 
September  1,  1987,  states  that  acquisition  programs  can  be 
designated  as  major  based  on  any  of  four  criteria:  Secretary  of 
Defense  interest,  joint  acquisition  of  a  system  by  two  or  more 
Services  or  with  other  nations,  estimated  system  costs  that 
exceed  $200  million  (FY  1980  dollars)  for  research  or  $1  billion 
(FY  1980  dollars)  for  procurement,  or  significant  congressional 
interest.  The  Secretary  of  Defense  relies  primarily  on  the  DAB 
for  advice  in  making  milestone  decisions  on  major  acquisition 
programs.  The  DAB,  which  is  chaired  by  the  Under  Secretary  of 
Defense  for  Acquisition,  provides  this  advice  by  evaluating  major 
acquisition  programs  that  are  approaching  milestone  decisions  and 
by  determining  whether  each  program  meets  the  criteria  that  are 
established  for  the  upcoming  milestone  decision.  The  DAB  then 
advises  the  Secretary  of  Defense  as  to  the  appropriate  action  to 
take  with  regard  to  the  system  under  review. 

The  Milestone  III  decision  approves  proceeding  with  either 
low-rate  initial  production  or  full-rate  production  and 
deployment.  For  major  acquisition  programs  approaching  the 
Milestone  III  decision  point,  the  DAB  evaluates  the  results  of 
completed  operational  tests  and  evaluations;  threat  validations; 
production  or  construction  cost  verifications;  affordability  and 
life-cycle  cost  estimates;  production  and  deployment  schedules; 
reliability,  maintainability,  and  plans  for  integrated  logistics 
support;  producibility  studies;  and  cost-effectiveness 
analyses.  If  the  magnitude  of  the  program  is  sufficiently  large 
and  the  period  between  the  beginning  of  low-rate  initial 
production  and  the  beginning  of  full-rate  production  is 
significantly  long,  there  may  be  a  need  for  a  program  review  or  a 
Milestone  IIIA  decision  before  the  Milestone  IIIB  decision. 

Another  key  factor  that  the  DAB  considers  before  granting  a 
production  request  is  the  proposed  rate  of  production.  U.S.C., 
title  10,  sec.  138,  stipulates  that  major  Defense  acquisition 
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programs  may  not  proceed  beyond  low-rate  initial  production 
until  certain  conditions  are  met.  First,  the  Military 
Departments  must  conduct  operational  test  and  evaluation 
(OT&E)  — /  of  the  components  that  are  being  acquired.  Second,  the 
Director,  OT&E,  must  independently  analyze  the  results  of  OT&E 
and  determine  whether  they  were  adequate  and  whether  the  results 
confirm  that  the  components  are  effective  and  suitable  for 
combat.  Third,  the  Director,  OT&E,  must  report  the  results  of 
his  independent  analysis  to  the  Secretary  of  Defense  and  to  the 
Senate  and  House  Committees  on  the  Armed  Services  and 
Appropriations . 

DoD  Directive  5000.3,  "Test  and  Evaluation,"  March  12,  1986, 
establishes  even  more  stringent  conditions  for  DoD  to  satisfy 
before  proceeding  beyond  low-rate  initial  production  on  a  major 
Defense  acquisition  program.  Not  only  does  the  Directive  require 
that  DoD  meet  the  conditions  stipulated  in  U.S.C.,  title  10, 
sec.  138,  but  paragraph  C.4.  of  the  Directive  states: 

Before  the  Milestone  III  production  decision,  the 
results  of  testing  (to  include  all  preproduction 
qualification  testing)  of  production-representative 
articles  shall  confirm  that  all  significant  design 
problems  have  been  identified,  that  solutions  to  these 
problems  are  available  and  that  the  items  or 
components  actually  tested  are  effective  and  suitable 
for  their  intended  use. 

Additionally,  paragraphs  C.3.a.  and  C.3.b.  of  the  Directive 
state: 


a.  Operational  testing  shall  be  accomplished  in  an 
environment  as  operationally  realistic  as  possible, 
including  threat  representative  hostile  forces. 
Typical  users  should  operate  and  maintain  the  system 
under  conditions  simulating  combat  stress  and 
peacetime  conditions. 


1/  Public  Law  98-94  defines  low-rate  initial  production  as 
"...the  production  of  a  system  in  limited  quantity  to  be  used  in 
operational  test  and  evaluation  for  verification  of  production 
engineering  and  design  maturity  and  to  establish  a  production 
base  prior  to  a  decision  to  proceed  with  production." 

— /  Sec.  138  defines  operational  test  and  evaluation  as  "(i)  the 
field  test,  under  realistic  combat  conditions,  of  any  item  of  (or 
key  component  of)  weapons,  equipment,  or  munitions  for  the 
purpose  of  determining  the  effectiveness  and  suitability  of  the 
weapons,  equipment,  or  munitions  for  use  in  combat  by  typical 
military  users:  and  (ii)  the  evaluation  of  the  results  of  such 
test . " 
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b.  The  primary  purpose  of  OT&E  is  to  ensure  that  only 
operationally  effective  and  suitable  systems  are 
delivered  to  the  operating  forces.  The  results  of 
OT&E  are  provided  to  the  appropriate  decision  makers 
for  decisions  on  system  production  and  fielding. 
Therefore,  OT&E  shall  be  structured  to  provide  inputs 
at  each  decision  point,  including  major  milestones. 

Internal  Controls 


We  reviewed  internal  controls  as  were  deemed  necessary  for 
the  stated  objectives.  Our  review  disclosed  weaknesses  in 
internal  controls  over  documentation  that  the  Military 
Departments  were  required  to  submit  to  the  DAB  and  over  system 
production  quantities  for  which  the  Military  Department  requested 
DAB  approval.  However,  for  the  reasons  discussed  in  the 
conclusion  to  this  report,  we  are  not  reporting  any  material 
internal  control  weaknesses. 

Prior  Audit  Coverage 

This  is  the  fifth  summary  audit  report  .issued  on  the 
Effectiveness  of  the  Defense  Acquisition  Board  Process.  The 
4  previous  reports  summarized  the  results  of  audits  on  29  major 
systems.  The  individual  system  reports  discussed  issues 
concerning  a  particular  system,  while  the  summary  reports 
discussed  systemic  issues  of  the  DAB  process  and  actions  required 
by  management  to  improve  the  acquisition  decisionmaking 
process.  The  four  summary  reports,  issued  between  October  1984 
and  July  1987,  are  briefly  described  in  Enclosure  2. 

Discussion 

Our  PY  1988  DAB  audits  disclosed  that,  for  the  most  part, 
the  Military  Departments  provided  the  DAB  with  the  documentation 
required  for  Milestone  III  production  decisions.  The  audits  did 
identify  key  documents  (cost  estimates,  decision  coordinating 
papers,  and  current  threat  assessments)  that  were  missing  or 
deficient  on  some  of  the  five  programs  that  we  audited.  However, 
since  such  instances  were  isolated,  and  we  advanced  recommenda¬ 
tions  to  correct  the  instances  during  the  audits,  we  will  not 
discuss  them  in  this  summary  report. 

The  most  prevalent  and  significant  condition  the  five  audits 
disclosed  was  that  the  Military  Departments  were  requesting 
approval  to  buy  systems  in  quantities  substantially  greater  than 
those  intended  by  Public  Law  and  DoD  Directives.  U.S.C., 
title  10,  sec.  138,  and  DoD  Directive  5000.3  limited  buys  on 
major  acquisition  programs  that  had  not  successfully  completed 
OT&E  to  the  number  of  systems  in  the  program  that  the  Military 

IT  Previously  known  as  the  Defense  Systems  Acquisition  Review 
Council. 
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Departments  needed  for  conducting  OT&E  and  for  establishing  a 
production  base.  However,  we  found  that  the  Military  Departments 
had  either  bought,  or  requested  approval  to  buy,  more  systems 
than  necessary  to  conduct  OT&E  and  to  establish  a  production 
base.  Furthermore,  the  Military  Departments  took  those  actions 
before  the  systems  successfully  completed  OT&E. 

Past  and  Proposed  Buys  of  Systems.  At  the  time  of  our 
audits,  the  quantities  of  systems  that  the  Military  Departments 
had  already  bought  and  had  requested  approval  to  buy  represented 
from  10.3  percent  to  100  percent  of  the  total  systems  planned  for 
the  five  programs  that  we  audited. 


NUMBER 

OF  SYSTEMS 

PERCENTAGE 

PROGRAM 

IN  PROGRAM  ON 

CONTRACT 

REQUESTED 

OF  PROGRAM 

AMRAAM 

24,320 

1,480 

1,270  y 

11.3 

SSN-21  submarine 

29 

1 

2 

10.3 

UHF  satellite 

10 

0 

69  y 

48  ±f 

100.0 

C-17  aircraft 

210 

2 

33.8 

T-45A  aircraft 

304 

12 

19.7 

1/  The  request  was  for  $400  million  to  initiate  long-lead 
procurement  action  on  Lot  III.  Total  Lot  III  procurement  cost 
was  estimated  at  $835  million. 

2/  The  request  consisted  of  40  aircraft  for  Lots  II  through  V 
and  long-lead  items  for  the  29  aircraft  in  Lot  VI.  The  estimated 
cost  associated  with  the  request  was  $10.3  billion. 

3/  The  request  consisted  of  $372.4  million  for  24  production 
aircraft,  $40.6  million  for  long-lead  items  for  an  additional 
24  aircraft,  and  $15.4  million  for  spare  parts. 


Neither  U.S.C.,  title  10,  sec.  138,  nor  DoD  Directive  5000.3 
stipulated  the  quantity  of  a  system  necessary  to  conduct  OT&E  and 
to  establish  a  production  base  for  major  acquisition  programs. 
However,  our  audits  concluded  that  the  Military  Departments 
requested  approval  to  buy  more  systems  than  section  138  and  DoD 
Directive  intended  on  all  five  of  the  programs  that  we  audited. 
For  example,  the  Military  Departments  requested  approval  to 
initiate  procurement  of  69  C-17  cargo  aircraft  and  48  T-45A 
trainer  aircraft.  Considering  the  quantity  previously  bought  for 
those  programs,  the  Military  Departments'  requests  were  in  effect 
asking  for  approval  to  buy  from  33.8  to  19.7  percent  of  the 
systems  in  the  programs  before  completion  of  OT&E.  Buying  such 
quantities  would  provide  more  aircraft  than  needed  to  conduct 
OT&E  and  to  establish  production  bases  for  the  programs.  The 
schedule  also  shows  that  the  Military  Departments'  requests 
provided  for  buying  100  percent  of  the  UHF  satellite  program. 
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10.3  percent  of  the  SSN-21  submarine  program,  and  11.3  percent  of 
the  AMRAAM  program.  We  concluded  that  such  quantities  exceeded 
the  intended  limitations  in  section  138  and  the  DoD  Directive. 

Operational  Test  and  Evaluation.  The  Military  Departments 
had  not  performed  OT&E  on  the  SSN-21  submarine  and  the  C-17  cargo 
aircraft.  Furthermore,  OT&E  performed  on  the  T-45A  trainer 
aircraft,  the  UHF  satellite,  and  the  AMRAAM  programs  had  not  yet 
confirmed  the  capabilities  of  those  programs.  For  example, 
requests  for  funds  for  the  T-45A  were  made  after  developmental 
test  and  evaluation  of  the  T-45A  had  shown  that  24  Part  I 
deficiencies  existed.  Part  I  deficiencies  are  those  that  affect 
safety  of  flight  or  mission  accomplishment.  Subsequent  OT&E  by 
Navy  test  pilots  confirmed  the  need  to  improve  the  aircraft's 
performance  in  order  to  meet  safety  and  mission  requirements. 
Also,  full  system  OT&E  results  would  not  be  available  for  the  UHF 
satellite  program  until  after  the  DAB  was  scheduled  to  hold  a 
Milestone  IIIB  (full-rate  production)  decision.  The  Navy  was 
seeking  approval  for  production  funding  for  10  satellites, 
100  percent  of  the  program. 

Effects  of  Requesting  Excessive  Acquisition  Quantities.  The 
Military  Departments'  requests  to  buy  excessive  systems  before 
the  completion  of  OT&E  did  not  have  significant  cost  consequences 
because  the  DAB  did  not  approve  the  Military  Departments' 
requests  for  the  numbers  of  systems  that  they  requested  approval 
to  buy.  In  each  of  the  five  system  reports,  we  recommended  that 
the  Under  Secretary  of  Defense  for  Acquisition  (USD(A))  reduce  or 
defer  the  numbers  of  systems  that  the  Military  Departments 
requested  approval  to  buy  until  the  systems  demonstrated  their 
combat  effectiveness  and  suitability  through  OT&E.  On  all  of  the 
recommendations,  the  USD(A)  reduced  or  deferred  the  requested 
production  decisions. 

Although  the  Military  Departments'  requests  did  not  have  any 
significant  cost  consequences  on  the  programs,  we  want  to 
emphasize  that  such  requests  pose  the  potential  to  expose  DoD  to 
unnecessary  cost  risks.  For  example,  if  the  USD(A)  had  not 
reduced  the  requested  acquisition  quantities,  the  Military 
Departments  would  have  been  able  to  invest  an  additional 
$12  billion  in  systems  that  had  not  yet  shown  their  combat 
effectiveness.  Such  an  investment  would  have  major  cost 
consequences  if  the  systems  do  not  prove  to  be  effective  or  if 
extensive  retrofits  are  required  to  make  the  systems  effective. 

Requesting  acquisition  quantities  that  exceed  the  intended 
limitations  set  forth  in  the  Public  Law  and  DoD  Directive  can 
also  unnecessarily  complicate  future  acquisition  decisions.  For 
instance,  if  the  requested  quantities  are  approved  and  bought  and 
the  contractors  establish  production  bases  to  satisfy  those 
quantities,  it  becomes  costly  to  reduce  future  buys  below  the 
previous  quantities  that  are  bought.  After  contractors  establish 
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their  production  bases  at  a  certain  level,  the  contractors  will 
normally  incur  additional  costs  if  they  have  to  scale  back  their 
production  bases. 

Factors  Contributing  to  the  Military  Departments'  Excessive 
Requests^  The  Military  Departments  were  able  to  request  approval 
to  buy  systems  in  quantities  that  exceeded  limitations  in  the 
Public  Law  and  DoD  Directive  because  there  was  no  requirement  for 
the  Military  Departments  to  determine  whether  their  requested 
quantities  were  consistent  with  provisions  in  the  Public  Law. 
Without  such  a  requirement,  the  Military  Departments  could 
request  any  quantities  within  their  budget  authorizations,  and 
that  is  what  the  Military  Departments  did.  For  example,  on  the 
T-45A  trainer  aircraft  program,  the  Navy  requested  approval  to 
buy  the  number  of  aircraft  in  the  acquisition  plan  despite  the 
poor  results  that  the  aircraft  achieved  in  developmental  testing 
and  initial  operational  test  and  evaluation.  Similarly,  the  Air 
Force  requested  approval  to  buy  the  C-17  aircraft  in  its  program 
although  no  operational  test  and  evaluation  had  been  performed. 
Furthermore,  the  Air  Force  had  not  prepared  a  formal  Test  and 
Evaluation  Master  Plan  prescribing  the  mission  scenarios  and  OT&E 
measures  necessary  to  fully  assess  the  mission  capability  of  the 
C-17  aircraft. 

Actions  Recommended  and  Taken  During  and  After  the  Audits. 
We  issued  five  audit  reports  recommending  various  actions  for 
management  to  take  on  the  programs  on  which  we  found  excessive 
acquisition  quantities.  The  USD(A)  responses  to  the 
recommendations  resulting  from  the  five  audit  reports  were 
mixed.  The  responses  ranged  from  almost  total  concurrence  on  the 
recommendations  on  the  T-45A  trainer  aircraft  program  to  almost 
total  nonconcurrence  on  the  SSN-21  submarine  program.  However, 
in  most  cases,  the  USD(A)  ultimately  took  actions  that  indicated 
agreement  with  our  recommendations. 

Other  actions  taken  by  OSD  have  also  had  an  impact  on  DoD 
management  of  major  weapon  system  acquisition  since  the  issuance 
of  our  five  audit  reports.  On  June  12,  1989,  the  Secretary  of 
Defense  released  his  Report  on  the  Defense  Management  Review 
(DMR).  A  major  objective  of  the  DMR  was  to  substantially  improve 
the  performance  of  the  Defense  acquisition  process.  The  DMR 
contains  wording  that  requires  the  DAB  to  rigorously  oversee 
major  system  acquisition.  In  addition,  in  a  February  22,  1990, 
memorandum  to  the  DAB  Committee  Chairman,  the  USD(A)  reemphasized 
the  need  to  ensure  that  all  major  acquisition  issues  are 
identified  and  that  required  documentation  was  complete  before 
formal  DAB  reviews.  We  believe  these  subsequent  actions  express 
a  concern  and  commitment  by  OSD  to  ensure  that  systems  will  meet 
operational  effectiveness  and  suitability  criteria  before 
proceeding  into  production.  Enclosure  3  presents  a  summary  of 
the  primary  recommendations  contained  in  our  five  system  reports 
and  the  actions  ultimately  taken  by  the  DSD(A) . 
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Congressional  action  taken  after  the  issuance  of  our 
five  reports  has  also  placed  new  emphasis  on  the  controls  to  be 
exercised  in  determining  the  quantity  of  systems  to  be  purchased. 
Section  802  of  the  National  Defense  Authorization  Act  for 
FY's  1990  and  1991  requires  that  DoD  complete  OT&E  before 
proceeding  beyond  low-rate  initial  production  on  major  systems. 
The  Act  also  redefined  low-rate  initial  production  and  provided 
exceptions  to  the  new  definition  for  Naval  vessels  and  satellite 
programs.  Details  of  the  congressional  action  are  presented  in 
Enclosure  4. 

Conclusion 


Because  of  the  extensive  changes  in  acquisition  management 
brought  about  by  Public  Law,  the  Defense  Management  Review 
Report,  and  changes  to  the  5000  series  of  Directives  and 
Instructions  since  our  five  audit  reports  were  issued,  no 
additional  recommendations  were  presented  in  this  report. 
However,  it  is  our  intent  to  concentrate  on  the  relationship 
between  the  degree  of  OT&E  completed  and  the  quantity  of  items 
authorized  for  production  during  our  planned  FY  1992  audit  of  the 
DAB  process. 

We  provided  a  draft  of  this  report  to  the  addressee  on 
November  29,  1990.  Because  the  report  contained  no 

recommendations,  no  comments  were  required  of  management,  and 
none  were  received.  Therefore,  we  are  publishing  this  report  in 
final  form.  Any  comments  to  this  final  report  should  be  received 
within  30  days  of  the  date  of  this  report.  This  report  contains 
no  potential  monetary  benefits. 

The  cooperation  and  courtesies  extended  to  the  audit  staff 
are  appreciated.  The  audit  team  members  are  shown  in 
Enclosure  6.  Please  contact  Mr.  Rayburn  H.  Stricklin,  Program 
Director,  at  (703)  614-3965  (AUTOVON  224-3965)  or  Mr.  Harry  L. 
Followell,  Project  Manager,  at  (703)  693-0400  (AUTOVON  223-0400), 
if  you  have  any  questions  concerning  the  report.  Copies  of  the 
final  report  will  be  distributed  to  the  activities  listed  in 
Enclosure  7.  . 

Robert  J \J Lieberman 
Assistant  Inspector  General 
for  Auditing 


Enclosures 


AUDIT  REPORTS  ISSUED  BY  THE 
ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING 
AND  INCLUDED  IN  THIS  SUMMARY  REPORT 


SYSTEMS  AUDITED 

REASON  FOR 
SELECTION 

REPORT 

NO. 

DATE 

ISSUED 

Navy 

SSN-21  Submarine  Program 

Follow-on  Ultra-High  Frequency 
Satellite  Program 

Milestone  III 
Milestone  III 

89-027 

89-041 

11/08/88 

12/22/88 

T-45A  Trainer  Aircraft  Program 

Milestone  IIIA 

89-096 

07/31/89 

Air  Force 

Advanced  Medium  Range  Air-to- 
Air  Missile  Program 

Milestone  IIIB 

88-200 

09/13/88 

C-17  Cargo  Aircraft  Program 

Milestone  IIIA 

89-067 

04/06/89 

ENCLOSURE  1 


SYNOPSES  OF  PRIOR  AUDIT  REPORTS 


Report  No.  85-009,  "Summary  Report  on  the  Audit  of  the 
Effectiveness  of  the  Defense  Systems  Acquisition  Review  Council 
(DSARC)  Process — Phase  I,"  October  24,  1984.  This  report 
summarized  the  results  of  audits  covering  six  systems  and 
presented  two  summary  findings  based  on  systemic  conditions  noted 
during  the  audit.  The  first  finding,  "DSARC  Documentation," 
reported  that  required  documentation  was  not  always  prepared  and 
provided  to  the  DSARC  within  established  time  frames.  The  second 
finding,  "Impact  of  Funding  Issues,"  reported  that  the  Military 
Departments  had  not  adequately  identified  either  the  source  or 
the  amount  of  funding  required  to  accomplish  the  system 
development  objectives  for  the  six  systems  reviewed. 

Report  No.  85-104,  "Summary  Report  on  the  Audit  of  the 
Effectiveness  of  the  Defense  Systems  Acquisition  Review  Council 
(DSARC)  Process — Phase  II,"  July  28,  1985.  This  report 
summarized  the  results  of  audits  covering  10  systems  and 
presented  2  summary  findings  based  on  systemic  conditions  noted 
during  the  review.  The  first  finding,  "Acquisition  Strategy," 
reported  that  required  Milestone  I  reviews  were  not  held  or 
planned  for  6  of  the  10  major  systems  included  in  the  audit.  The 
second  finding,  "Justification  for  a  Major  System  New  Start 
Document,"  reported  that  the  new  start  documents  for  five  major 
systems  did  not  adequately  present  all  necessary  information,  as 
required  by  the  5000  series  of  DoD  Directives. 

Report  No.  87-166,  "Summary  Report  on  the  Audit  of  the 
Effectiveness  of  the  Defense  Systems  Acquisition  Review  Council 
(DSARC)  Process — Phase  III,"  June  3,  1987.  This  report 
summarized  the  results  of  audits  covering  six  systems  and 
presented  two  summary  findings  based  on  systemic  conditions  noted 
during  the  review.  The  first  finding,  "Operational  Thresholds," 
reported  that  program  managers  had  not  provided  adequate 
operational  performance  thresholds  at  the  Milestone  II  decision 
point  to  serve  as  overall  operational  objectives  to  be  met  before 
receiving  production  approval  from  the  Secretary  of  Defense.  The 
second  finding,  "Major  Changes  to  Acquisition  Programs,"  reported 
that  the  Navy  and  Air  Force  restructured  the  acquisition 
strategies  for  two  major  programs  without  first  obtaining  the 
permission  of  the  Secretary  of  Defense. 

Report  No.  87-193,  "Summary  Report  on  the  Audit  of  the 
Effectiveness  of  the  Defense  Systems  Acquisition  Review  Council 
(DSARC)  Process — FY  1986,"  July  17,  1987.  This  report  summarized 
the  results  of  audits  covering  seven  individual  systems  but  did 
not  disclose  any  new  systemic  issues  that  would  require  policy 
changes  and  therefore  did  not  contain  any  new  findings  and  recom¬ 
mendations.  However,  it  did  reaffirm  our  prior  observations  that 
more  rigorous  enforcement  of  existing  acquisition  policy  was 
needed  and  that  recommendations  contained  in  prior  summary 
reports  remained  valid  and  unaccomplished. 
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ACTIONS  RECOMMENDED  AND  TAKEN 
DURING  AND  AFTER  THE  AUDIT 


We  issued  five  audit  reports  recommending  various  actions 
for  management  to  take  on  the  programs  on  which  we  found 
excessive  acquisition  quantities.  Our  specific  recommendations 
and  actions  taken  by  the  Under  Secretary  of  Defense  for 
Acquisition  (USD[A])  are  summarized  below. 

Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) .  We 
recommended  that  the  USD(A)  maintain  low-rate  production  with  Lot 
III  of  the  AMRAAM  program  until  the  Air  Force  provides  favorable 
operational  test  results.  On  July  20,  1988,  USD(A)  nonconcurred 
with  the  recommendation  because,  at  the  time  of  the  response,  the 
Defense  Acquisition  Board  (DAB)  had  not  made  a  decision  on  the 
Lot  III  request.  In  addition,  the  USD(A)  stated  that  the  Air 
Force's  Lot  III  request  for  long-lead  procurement  authority  for 
1,270  missiles  was  a  de  facto  low-rate  production  request  because 
it  represented  a  reduction  from  2,000  missiles  in  1985.  However, 
the  USD(A)  subsequently  deferred  the  decision  of  the  Air  Force's 
Lot  III  request  until  after  a  September  1988  DAB  program 
review.  A  subsequent  program  review  continued  to  restrict 
quantities  because  of  poor  test  results. 

SSN-21  Submarine.  We  recommended  that  the  USD(A) 
schedule  a  DAB  program  review  for  the  SSN-21  submarine  during 
FY  1995  when  more  complete  development  and  testing  results  would 
be  available.  We  also  recommended  that  the  Chief  of  Naval 
Operations  provide  the  results  of  an  assessment  of  the  military 
worth  of  the  SSN-21  submarine  at  the  time  of  initial  operating 
capability  and  10  years  into  the  future.  Although  both  the 
USD(A)  and  the  Navy  nonconcurred  with  the  recommendations,  the 
Secretary  of  Defense  deferred  the  Navy's  request  for  advance 
procurement  funding  for  two  additional  submarines  until  the  third 
quarter  of  FY  1990.  The  deferral  was  made  to  allow  time  to 
complete  and  review  an  operational  effectiveness  and  suitability 
assessment  and  a  live  fire  test  report.  Subsequently,  the  House 
Committee  on  Appropriations,  in  its  Report  on  the  Department  of 
Defense  Appropriations  Bill  —  1990,  required  the  Navy  to  perform 
a  "dynamic  mission  analysis"  on  the  initial  operational 
capability  plus  10  years. 

Follow-on  Ultra-High  Frequency  (UHF)  Satellite.  We 
recommended  that  the  USD(A)  withhold  the  full-rate  production 
decision  on  the  UHF  satellite  until  operational  and  technical 
effectiveness  measures  have  been  established,  an  assessment  of 
system  effectiveness  and  suitability  by  subsystem  and  component 
testing  has  been  completed,  and  the  Navy  provides  justification 
for  the  number  of  satellites  actually  required.  Comments 
received  from  the  USD(A)  indicated  general  concurrence  with  our 
recommendations  and  the  Acquisition  Decision  Memorandum  required 
the  Navy  to  prepare  an  updated  Test  and  Evaluation  Master  Plan. 
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ACTIONS  RECOMMENDED  AND  TAKEN 

DURING  AND  AFTER  THE  AUDIT  (continued) 


C-17  Cargo  Aircraft.  We  recommended  that  the  USD(A) 
reduce  the  scope  of  the  DAB  Milestone  IIIA  review  for  the  C-17 
cargo  aircraft  program,  planned  for  October  1990,  to  the  limited 
production  rate  established  at  the  December  1988  program  review 
(10  units  per  year),  until  the  planned  March  1993  Milestone  IIIB 
review  is  held.  In  addition,  we  recommended  that  the  USD(A) 
require  the  Air  Force  to  revise  the  C-17  cargo  aircraft  Test  and 
Evaluation  Master  Plan  to  include  valid  operational  effectiveness 
measures  and  thresholds  and  a  testing  phase  dedicated  to 
evaluating  the  operational  mission  effectiveness  of  the 
aircraft.  The  USD(A)  nonconcurred  with  the  recommendation  to 
reduce  the  scope  of  the  Milestone  IIIA  review  to  10  aircraft. 
However,  the  USD(A)  subsequently  authorized  procurement  of  only 
10  aircraft  and  long-lead  items  for  10  more  aircraft  instead  of 
the  requested  40  aircraft  and  long-lead  items  for  29  more 
aircraft . 


T-45A  Trainer  Aircraft.  We  recommended  that  the  USD(A) 
schedule  a  DAB  Program  Review  of  ongoing  T-45A  trainer  aircraft 
performance  corrective  actions  and  the  Navy's  change  in  the 
program  baseline  before  release  of  FY  1989  funds.  We  also 
recommended  that  the  USD(A)  schedule  a  DAB  Milestone  IIIA  review 
for  the  T-45A  trainer  aircraft  program  before  the  release  of 
FY  1991  funds.  The  Milestone  IIIA  review  would  be  scheduled 
contingent  upon  successful  completion  of  ongoing  developmental 
and  operational  testing.  The  USD(A)  and  the  Comptroller,  DoD, 
concurred  with  the  recommendations. 

On  June  12,  1989,  just  before  our  final  systems  report  was 
issued,  the  Secretary  of  Defense  took  a  major  action  that  was 
intended  to  discourage  the  Military  Departments  from  requesting 
excessive  acquisition  quantities  before  completion  of  operational 
test  and  evaluation.  Specifically,  the  Secretary  of  Defense 
released  his  Report  on  the  Defense  Management  Review  (DMR) ,  which 
set  forth  the  Department  of  Defense's  plan  to: 

-  fully  implement  the  Packard  Commission's 
recommendations , 

-  substantially  improve  the  performance  of  the  Defense 
acquisition  system,  and 

-  more  effectively  manage  the  Department  of  Defense  and 
its  resources. 
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ACTIONS  RECOMMENDED  AND  TAKEN 

DURING  AND  AFTER  THE  AUDIT  (continued) 


In  addressing  the  DAB's  responsibilities,  the  DMR  Report  added 
emphasis  to  the  DAB  process  by  stipulating  that: 

The  DAB  will  rigorously  oversee  major  systems 
acquisition,  to  ensure  that  the  acquisition  process  is 
managed  in  a  manner  consistent  with  DoD  policy.  That 
policy  will  define  minimum  required  accomplishments, 
and  permit  additional  program  specific  exit  criteria 
to  be  established  by  the  USD(A),  at  each  Milestone  in 
a  systems  life.  The  paramount  objective  of  the  USD(A) 
will  be  to  discipline  the  acquisition  system  through 
review  of  major  programs  by  the  DAB. 

We  believe  that  the  wording  in  the  DMR  Report  expresses  the 
commitment  of  the  USD(A)  to  have  program  offices  establish 
meaningful  cost,  schedule,  and  performance  criteria  and  ensure 
that  the  systems  meet  that  criteria  before  being  allowed  to  move 
from  one  acquisition  phase  to  the  next. 

In  addition,  in  a  February  22,  1990,  memorandum  to  DAB 

Committee  Chairmen,  the  USD(A)  provided  guidance  on  improving  the 
acquisition  review  process.  The  memorandum,  endorsed  by  the 
Service  Acquisition  Executives,  established  a  formal  Systems 
Committee  preplanning  meeting  6  months  before  the  actual  DAB 
review  to  identify  major  issues,  emphasized  the  prompt  submission 
of  required  documentation,  required  that  all  documentation  be 
reviewed  by  OSD  staff  members  to  determine  if  all  major  issues 
had  been  resolved,  and  required  the  OSD  Cost  Analysis  Improvement 
Group  to  independently  review  the  program  costs  established  by 
the  respective  program  office  and  the  Military  Department's 
Independent  Cost  Analysis  teams.  The  USD(A)  memorandum  also 
stipulated  that  the  Systems  Committee  meeting  and  subsequent  DAB 
review,  would  be  postponed  if  documentation  was  not  submitted  in 
a  timely  manner.  The  overall  objective  of  these  actions  was  to 
streamline  the  DAB  process. 
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RELATED  CONGRESSIONAL  ACTIONS 


After  our  audits.  Congress  amended  U.S.C.,  title  10, 
section  138,  with  requirements  that  placed  more  control  over  the 
quantity  of  systems  that  the  Military  Departments  can  buy  before 
the  completion  of  operational  test  and  evaluation.  Section  802 
of  the  National  Defense  Authorization  Act  for  FY's  1990  and  1991 
amended  title  10  with  laws  that  stipulated  that  DoD  must  complete 
initial  operational  test  and  evaluation  before  proceeding  beyond 
low-rate  initial  production  on  major  acquisition  programs.  Also, 
for  a  new  major  acquisition  program  or  system,  section  803  of  the 
Act  defined  low-rate  initial  production  as  the  quantity 
necessary  s 

(1)  to  provide  production-configured  or 

representative  articles  for  operational  tests  pursuant 
to  section  2399  of  this  titLe; 

(2)  to  establish  an  initial  production  base  for  the 
system;  and 

(3)  to  permit  an  orderly  increase  in  the  production 
rate  for  the  system  sufficient  to  lead  to  full-rate 
production  upon  the  successful  completion  of 
operational  testing. 

In  addition,  the  Act  established  new  low-rate  initial  production 
guidelines  for  Naval  vessel  and  satellite  programs.  The  Act 
stipulates  the  following. 

(1)  .  .  .  low-rate  initial  production  is  production  of 
items  at  the  minimum  quantity  and  rate  that  (A) 
preserves  the  mobilization  production  base  for  that 
system,  and  (B)  is  feasible,  as  determined  pursuant  to 
regulations  prescribed  by  the  Secretary  of  Defense. 

(2)  .  •  .  the  Secretary  of  Defense  shall  submit  to 

Congress  a  report  providing  -  (A)  an  explanation  of 
the  rate  and  quantity  prescribed  for  low-rate  initial 
production  and  the  considerations  in  establishing  that 
rate  and  quantity;  (B)  a  test  and  evaluation  master 
plan  for  that  program;  and  (C)  ...  an  acquisition 

strategy  for  that  program  that  has  been  approved  by 
the  Secretary,  to  include  the  procurement  objectives 
in  terms  of  total  quantity  of  articles  to  be  procured 
and  annual  production  rates. 
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RELATED  CONGRESSIONAL  ACTIONS  (continued) 


Section  803  also  requires  DoD  to  determine  how  many  systems  on 
major  acquisition  programs  should  be  procured  as  low-rate  initial 
production  when  the  Milestone  II  (full-scale  engineering 
development)  decision  is  made.  In  addition,  the  section  requires 
the  Secretary  of  Defense  to  include  the  low-rate  initial 
production  quantities  in  the  first  Selected  Acquisition  Report 
that  is  prepared  after  the  quantities  are  determined.  Also,  any 
changes  to  the  quantities  must  be  approved  by  the  official  that 
originally  established  them. 
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ACTIVITIES  VISITED  OR  CONTACTED 


Office  of  the  Secretary  of  Defense 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Washington,  DC 

Office  of  the  Comptroller  of  the  Department  of  Defense, 
Washington,  DC 

Office  of  the  Director,  Operational  Test  and  Evaluation, 
Washington,  DC 

Office  of  the  Director,  Program  Analysis  and  Evaluation, 
Washington,  DC 

Office  of  the  Director  (Command,  Control  and  Communications), 
Joint  Chiefs  of  Staff,  Washington,  DC 

Department  of  the  Navy 

Office  of  the  Chief  of  Naval  Operations,  Washington,  DC 
Headquarters,  Naval  Technical  Intelligence  Center,  Washington,  DC 
Headquarters,  Naval  Telecommunications  Command,  Washington,  DC 
Headquarters,  Naval  Air  Test  Center,  Patuxent  River,  MD 
Follow-on  UHF  Satellite  Program  Office,  PM-146,  Washington,  DC 
Fleet  Satellite  Communications  Office,  Los  Angeles,  CA 
T-45TS  Training  System  Office,  Washington,  DC 
SSN-21  Program  Office,  PM-350,  Washington,  DC 

Department  of  the  Air  Force 

Office  of  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition),  Washington,  DC 

Office  of  the  Deputy  Chief  of  Staff,  Plans  and  Operations, 
Washington,  DC 

Office  of  the  Assistant  Chief  of  Staff,  Systems  for  Command, 
Control,  Communications  and  Computers,  Washington,  DC 
Headquarters,  Air  Force  Operational  Test  and  Evaluation  Center, 
Kirtland  Air  Force  Base,  NM 

C-17  Program  Office,  Aeronautical  Systems  Division,  Wright- 
Patterson  Air  Force  Base,  OH 

Advanced  Medium  Range  Air-to-Air  Missile  Joint  Systems  Program 
Office,  Eglin  Air  Force  Base,  FL 

Defense  Agency 

Headquarters,  Defense  Intelligence  Agency,  Washington,  DC 
Non-Department  of  Defense 

Headquarters,  Central  Intelligence  Agency,  Arlington,  VA 
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AUDIT  TEAM  MEMBERS 


Donald  E.  Reed,  Director,  Acquisition  Management  Directorate 

Rayburn  H.  Stricklin,  Program  Director 

Harry  L.  Followell,  Project  Manager 

William  Hopple,  Team  Leader 

James  Cochrane,  Auditor 

Cordelia  Grace-Scott,  Auditor 

Audrey  M.  Spear,  Auditor 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition 
Under  Secretary  of  Defense  for  Policy 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications 
and  Intelligence) 

Assistant  Secretary  of  Defense  (Production  and  Logistics) 
Director,  Operational  Test  and  Evaluation 

Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial  Management) 

Auditor  General,  U.S.  Army  Audit  Agency 

Department  of  the  Navy 

Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management) 

Assistant  Secretary  of  the  Navy  (Research,  Development  and 

Acquisition ) 

Director,  Naval  Audit  Service 
Department  of  the  Air  Force 
Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 

Comptroller ) 

Auditor  General,  Air  Force  Audit  Agency 
Defense  Agency 

Director,  Defense  Logistics  Studies  Information  Exchange 
Non-DoD  Activities 


Office  of  Management  and  Budget 

U.S.  General  Accounting  Office,  NSIAD  Technical  Information 
Center 

Congressional  Committees: 

Senate  Subcommittee  on  Defense,  Committee  of  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Ranking  Minority  Member,  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Ranking  Minority  Member,  Committee  on  Appropriations 

House  Committee  on  Armed  Services 

House  Committee  on  Government  Operations 

House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 


ENCLOSURE  7 


INTERNET  DOCUMENT  INFORMATION  FORM 


A .  Report  Title:  Summary  Report  on  the  Audit  of  the  Effectiveness  of  the 
Defense  Acquisition  Board  Process — FY 1988 

B.  DATE  Report  Downloaded  From  the  Internet:  08/08/00 


C.  Report's  Point  of  Contact:  (Name,  Organization,  Address,  Office 
Symbol,  &  Ph  #):  OAIG-AUD  (ATTN:  AFTS  Audit  Suggestions) 

Inspector  General,  Department  of  Defense 
400  Army  Navy  Drive  (Room  801) 

Arlington,  VA  22202-2884 


D.  Currently  Applicable  Classification  Level:  Unclassified 

E.  Distribution  Statement  A:  Approved  for  Public  Release 

F.  The  foregoing  information  was  compiled  and  provided  by: 

DTIC-OCA,  Initials: _ VM _ Preparation  Date  08/08/00 


The  foregoing  information  should  exactly  correspond  to  the  Title,  Report  Number,  and  the  Date  on 
the  accompanying  report  document.  If  there  are  mismatches,  or  other  questions,  contact  the 
above  OCA  Representative  for  resolution. 


